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Abstract. The paper concerns the investigation of nonconvex and nondiffer- 
entiable integral functionals on general Banach spaces, which may not be reflexive 
and/or separable. Considering two major subdifferentials of variational analysis, 
we derive nonsmooth versions of the Leibniz rule on subdifferentiation under the in¬ 
tegral sign, where the integral of the subdifferential set-valued mappings generated 
by Lipschitzian integrands is understood in the Gelfand sense. Besides examining 
integration over complete measure spaces and also over those with nonatomic mea¬ 
sures, our special attention is drawn to a stronger version of measure nonatomicity, 
known as saturation, to invoke the recent results of the Lyapunov convexity theorem 
type for the Gelfand integral of the subdifferential mappings. The main results are 
applied to the subdifferential study of the optimal value functions and deriving the 
corresponding necessary optimality conditions in nonconvex problems of stochastic 
dynamic programming with discrete time on the infinite horizon. 
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1 Introduction 

This present study belongs to the area of infinite-dimensional variational 


analysis and its applications to which Professor Antonino Maugeri has made 


well-recognized contributions; see, e.g., [29] and the references therein. 

The main object of our investigation in this paper is the class of integral 
functionals of the type 



( 1 . 1 ) 


^Department of Mathematics, Wayne State University, Detroit, Michigan 48202, USA 
(boris@math.wayne.edu). Research of this author was partly supported by the USA Na¬ 
tional Science Foundation under grants DMS-1007132 and DMS-1512846 and by the USA 
Air Force Office of Scientific Research under grant No. 15RT0462. 

^Department of Economics, Hosei University, Tokyo 194-0298, Japan 
(nsagara@hosei.ac.jp). Research of this author was partly supported by JSPS KAKENHI 
Grant Number 26380246 from the Ministry of Education, Culture, Sports, Science and 
Technology, Japan. 


1 



generated by some integrand (/?: X x O —^ M := (—cx),oo], which is defined 
on a Banach space X and a finite measure space Our major goal is to 
establish new results of subdifferential calculus for dni related to nonconvex 
subdifferential versions of the classical Leibniz rule on differentiation under 
the integral sign. Subdifferentiation of integral functionals of type dEU) and 
appropriate versions of the Leibniz rule were developed in [211 Sni He] for 
convex integrands and then partially extended in p dni ESI Ea EH ES] to 
some nonconvex settings. In particular, the inclusion 

( 9 ^ J (f{-,t)dt^{x) C cl j d(p{x,t)dt ( 1 - 2 ) 

was established in m Lemma 6.18] for the basic/limiting subdifferential by 
Mordukhovich m, where (/?: X x [0,1] ^ M is Lebesgue measurable in t and 
locally Lipschitzian in x around x with a summable modulus on [0,1], where 
X is reflexive and separable, where the closure “cl” is taken in the norm 
topology of X*, and where the integral is understood in the Bochner sense. 
The obtained formula (|1.2I) was applied in m Chapter 6] to the derivation 
of necessary optimality conditions of the extended Euler-Lagrange type for 
generalized Bolza problems governed by evolution inclusions. 

The original motivation for the current paper was to develop appropriate 
versions of o in order to achieve the following goals: 

(a) Get an extension of (|1.2I) to more general classes of measure spaces. 

(b) Obtain conditions allowing us to dismiss the closure operation in the 
suitable counterparts of (11.21) . 

(c) Avoid or relax the reflexivity and separability assumptions on the 
Banach space X for the validity of (11.21) and its extensions. 

In the rest of this paper the reader can find a resolution of the issues 
listed in (a)-(c) together with additional results in these directions. The 
crucial ingredients of the progress achieved include the following; 

(i) The usage of the Gelfand integral instead of the Bochner one in the 
case of mappings on complete measure spaces with values in a space, which 
is topologically dual to an arbitrary Banach space. Note that both these 
integrals agree in the setting of 1311 Lemma 6.18]; in fact, in any separable 
Asplund space X. Our achievements in this direction are based on a quite 
recent progress in studying the Gelfand integrand of set-valued mappings 
made by Gascales et al. in [Hi; see Section 2 for more details. 

(ii) The usage of the measure nonatomicity and the Lyapunov convexity 
theorem, which allows us to derive an extension of to general measure 
spaces 0 enjoying this property and (nonreflexive and nonseparable) Asplund 
spaces X with the replacing the norm closure “cl” on the right-hand side of 

by the weak* closure; the latter in not needed if X is reflexive. 

(iii) The usage of saturation (or “super-atomlessness”) property of mea¬ 
sure spaces, which is equivalent to the enhanced version of the Lyapunov 
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convexity theorem for the Gelfand integral of set-valued mapping without 
the closure operation on the right-hand side of (ll.2p ; see [26l |38l US] • 

It is worth noting that the aforementioned results obtained in the vein 
of (jl.2l) are new even in the case of the Bochner integral in reflexive and 
separable Banach spaces X. Observe also that, in the process of implement¬ 
ing our approach, we establish similar inclusions for the Gelfand integral 
(more precisely, its t(;*-integral version for convex-valued multifunctions) of 
the generalized gradient by Clarke [lOj . which is the convexification of the 
limiting subdifferential in the case of Asplund spaces. Moreover, it is shown 
below that the Gelfand integral counterpart of ca) holds for the general¬ 
ized gradient in any Banach space with omitting the closure operation on 
the right-hand side in the case of an arbitrary complete measure space fl, 
even without imposing the saturation requirement on the measure. 

The obtained subdifferential formulas for integral functionals are further 
applied to problems of stochastic dynamic programming (DP) for inhnite- 
dimensional discrete-time systems on the infinite horizon, which are impor¬ 
tant, e.g., in macroeconomic modeling. We first derive the Bellman equation 
for the intrinsically nonsmooth random optimal value function in such sys¬ 
tems, then justify its local Lipschitz continuity under natural assumptions 
and evaluate its sub differentials. These results readily imply necessary opti¬ 
mality conditions in the stochastic dynamic programs under consideration. 

The rest of the paper is organized as follows. Section 2 is devoted to 
the study of the Gelfand integral and its t(;*-integral modification for set¬ 
valued mappings/multifunctions with values in duals to Banach spaces. We 
provide a brief overview of some basic facts needed in the sequel and obtain 
a new result on the convexity of the Gelfand integral of set-valued mappings 
defined on nonatomic measure spaces with no separability assumption on 
the Banach space in question. 

Section 3 contains our major results on subdifferentiation of integral 
functionals of type 0 . First we derive a generalized gradient version of the 
Leibniz formula in the case of integrands ip{x,uj) defined on the product of 
an arbitrary Banach space and a complete measure space, provided that the 
generalized gradient of y? in a: is t(;*-scalarly measurable. Effective sufficient 
conditions for the latter properties are discussed in detail. Then we proceed 
with several versions of the Leibniz rule in the case of Gelfand integral 
functionals associate with the limiting subdifferential. In this way we resolve 
the issues discussed above in (a)-(c). 

The last two sections concern applications of the obtained subdiffer¬ 
ential results for integral functionals to problems of stochastic DP in Ba¬ 
nach spaces. In Section 4 we describe a general stochastic DP problem for 
discrete-time systems on the infinite horizon and show that its optimal value 
function is a unique solution to the corresponding Bellman equation in an 
integral form with respect to a transition probability measure induced by 
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the Markov decision process. Proving the local Lipschitz continuity of the 
value function under appropriate assumptions, we employ the results of Sec¬ 
tion 3 to evaluate the corresponding subdifferentials of the value function 
and derive necessary optimality conditions in the stochastic DP problem. 

Throughout the paper we use the standard terminology and notation 
recalled in the places they first appear in the text below. 

2 Gelfand Integrals of Set-Valued Mappings 

We start this section by recalling the notion of the rc^-integral for a function 
on a measurable space with values in dual Banach spaces. Let (D, S, fi) be 
a complete finite measure space, and let X be a real Banach space with the 
dual system {X,X*), where X* is the norm/topological dual of X, while 
w* indicates the usage of the weak* topology on X* in what follows. A 
function / : D —>■ X* is w*-scalarly measurable if the scalar function (/(•), x) 
is measurable for every x £ X. We say that a w;*-scalarly measurable 
function is w*-scalarly integrable if (/(•), x) is integrable for every x € X. 
Further, we say that a r(;*-scalarly measurable function / is w*-integrable 
(or Gelfand integrable) over a given set A G S if there exists € X* such 
that (x^,x) = jj^{f{ijj),x)dyi for every x € X. The element x^, which is 
unique by the separation theorem, is called the Gelfand or t(;*-integral of 
/ over A and is denoted by fd^ . If A = D, we omit indicating the 
set in the integral sign. Note that every r(;*-scalarly integrable function is 
w*-integrable as shown in [H Theorem 11.52]. While we do not distinguish 
between the Gelfand integral and the t(;*-integral of single-valued mappings, 
we do it for set-valued mappings/multifunctions; see below. 

Definition 2.1 (Gelfand integral of general set-valued mappings). 

Let (D, S, fi) be a complete finite measure space, and let X be a Banach space. 
Given a set-valued mapping P: D ^ X*, we denote by the eollection of 
all the w*-integrable seleetors of T on LI, i.e., such w*-integrable funetions 
/: D — >■ X* that f{uj) G r(w) for a.e. uj G LI. Then the Gelfand integral 
of the multifunction P over the measure space D is defined by 

/(P) = J Tdtr:=y fdp.\fGSy (2.1) 

The first question arising in the study of the Gelfand integral m is 
about nonemptiness of the set /(P). The answer is affirmative in a rather 
broad setting due to a nice selection theorem from [8] . Although our stand¬ 
ing framework in this paper is the general class of Banach spaces, this result 
and some other ones require the Asplund property of the space in question. 
Recall that a Banach space X is Asplund if every convex continuous function 

1/ —>■ M defined on an open convex set U C X is Frechet differentiable 
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on a dense subset of U. This class of Banach spaces is sufficiently large in¬ 
cluding, in particular, any space with a Frechet differentiable bump function 
(hence any space admitting an equivalent norm Frechet differentiable off the 
origin, i.e., a Frechet smooth renorm, and therefore every refiexive space), 
any space with a separable dual, and any space X whose dual X* is weakly 
compactly generated meaning that there exists a weakly compact subset of 
X* whose linear span in norm sense. There are many useful characteriza¬ 
tions of Asplund spaces; among the most remarkable ones we mention that 
X is Asplund if and only if every closed separable subspace of X* has a 
separable dual. It is relevant to mention that any separable Asplund space 
admits a Frechet smooth renorm. We refer the reader to the book m and 
the bibliography therein for all these and other facts on Asplund spaces. 

An appropriate definition of measurability for set-valued mappings is 
needed to be recalled in order to list conditions ensuring the nonemptiness 
of the Gelfand integral ()2.ip . Denoting by 

s{x,C) := sup(x*,x), x^X, (2.2) 

x*&C 

the support function of a nonempty set C C A, we say that a set-valued map¬ 
ping F: D X* is w*-scalarly measurable if its support function s(x, F): D ^ 
M is measurable for every x ^ X. Further, a multifunction F is said to be in- 
tegrably bounded if there exists G such that sup 2 ,»er(^) ||x*|| < if(co) 

for every uj G kl. Now we are ready to formulate the aforementioned result 
on nonemptiness of the Gelfand integral m- 

Proposition 2.2 (nonemptiness of the Gelfand integral). Let (D, S, fi) 

be a complete finite measure space, and let X be an Asplund space. Assume 
that F: D ^ X* is an integrably bounded and w*-scalarly measurable mul¬ 
tifunction with w*-closed values. Then the Gelfand integral of the 

mapping T is a nonempty subset of X*. 

Proof. Since F has w;*-compact values and its integral boundedness guaran¬ 
tees the ?/;*-integrability of w;*-scalarly measurable selectors, the nonempti¬ 
ness of the Gelfand integral follows from definition (j2.ip and the measurable 
selection result from [HJ Corollary 3.11], which ensures the existence of w*- 
scalarly measurable selectors of F under the assumptions made. □ 

At this stage, it makes sense to compare the Gelfand and Bochner inte¬ 
grals of single-valued and set-valued mappings. 

Remark 2.3 (comparison with the Bochner integral). Denote by 
Borel(A*, w*) the Borel cr-algebra of X* generated by the weak* topology 
and by Borel(A*, H-H) the Borel cr-algebra of X* generated by the dual norm. 
Recall that a function / : D —>■ X* is strongly (or Bochner) measurable if it 
is a pointwise A*-norm limit of a sequence of A*-valued simple functions. 
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It is well known that if X is a separable Banach space, then X* is a locally 
convex Suslin space under the weak* topology and / is r(;*-scalarly measur¬ 
able if and only if it is Borel(X*, ?/;*)-measurable; see [451 Theorem 1] or [461 
Lemma 2], Furthermore, if X* is separable, then Borel(X*, w*) agrees with 
Borel(X*, II • II) by [HI Corollary 2 from Part I, Ch. II]. It implies, by taking 
into account that equality, the strong, Borel(X*, w*)-, and Borel(X*, || • ||)- 
measurabilities coincide. Combining these facts allows us to deduce that a 
Gelfand integrable function / satisfying f \\f{uj)\\dfj, < oo is also Bochner in- 
tegrable if X is Asplund and separable. Therefore, defining similarly to (12.11) 
the Bochner integral of a set-valued mapping T: 12 ^ X* , we conclude that 
the Gelfand and Bochner integrals of T agree in separable Asplund spaces. 

To proceed with studying the integral (|2.ip of multifunctions T: 12 ^ 
X*, recall that T is w*-scalarly integrable if its support function s(a:,r) is 
integrable for every x € X. The next proposition is useful in what follows. 

Proposition 2.4 (supremum representation for the Gelfand inte¬ 
gral). Let X be an Asplund space, (12,11,//) be a complete finite measure 
space, and T : 12 X* be a Gelfand integrable, w*-scalarly integrable multi¬ 
function with w*-compact values. Then we have 


/ s{x,T)dfi = sup / {f{uj),x)dpL for every x £ X. (2-3) 

Proof. For any x £ X we define the multifunction Fj,: 12 ^ X* with w*- 
compact values in X* by 

rx(w) := {x* £ F(a;) | {x*,x) = s(x,r(a;))}. 

As observed in (HJ Remark 4.4], F^, is t(;*-scalarly measurable, and hence it 
admits a r(;*-scalarly measurable selector fx by [HI Corollary 3.11]. Thus 

s{x,T{uj)) = {fx{u}),x) for a.e. a; € 12. 

Integrating both sides of the above equality tells us that 




/ {fx{uj),x)diJ. < sup 
J /65l 


j {f{^),x)dfi. 


On the other hand, it follows from (12.2p that s(a;, F(a;)) > {x, f{uj)) for every 
/ € a.e. io £ Ll. Integrating both sides of this inequality yields 


/ s{x, r)d/i > sup 
/65i 


j (/(w),a:)d/i. 


which readily verifies the claimed equality (12.3p . 


□ 
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Next we describe an alternative approach to constructing a Gelfand- 
type integral of confea;-valued multifunctions, which agrees with the Gelfand 
integral from Definition 12.11 in the case of separable spaces X while it may 
be a bit different from (12.ip in the absence of separability. To proceed, 
denote by cb*" T the multifunction defined by taking the w*-closure of the 
convex hull of r(ti;) for every w € D. It follows from (|2.2H that s(a;,r(a;)) = 
s{x,c6^ r(a;)) for every x ^ X and w € D, and that T is r(;*-scalarly 
measurable (resp. w^-scalarly integrable) if and only if so is cb’" T. We say 
that / : D ^ X* is a w*-almost selector of T if, for every x (z X 

{f{uj),x) < s{x,T{io)) for a.e. a; G D, 

where the corresponding ^-null set may depend on x. Similarly to the set 
in Definition 12.11 denote by 5pthe collection of r(;*-integrable, w^-almost 
selectors of a multifunction T : D ^ X*. We clearly have the inclusion 
<Sp C 5p. If furthermore X is separable and T has w*-compact and 
convex values, then iSp = as shown in [8l Proposition 2.3]. 

Definition 2.5 (u)*-integral of convex-valued multifunctions). Let 

(D, S, p) be a complete finite measure space, and let X he Banach. We say 
that a w*-scalarly measurable multifunction T: D ^ X* with w*-compact 
and convex values is r(;*-iNTEGRABLE if it is w* -scalarly integrable and for 
every A gT, there is a w*-compact and convex set Ca C X* such that 

s{x,Ca) = / s{x,T)dp, for every x G X. 

Ja 

Then the set Ca is called the u;*-integral of P over A and denoted by 
w*-J^Tdp., where the set indication is omitted when X = D. 

The classical separation theorem tells us that the tt)*-integral w*-f^rdp, 
is uniquely determined for every X G S. Some useful facts on r(;*-integrability 
of multifunctions are summarized in the next proposition taken from [HI The¬ 
orem 2.7 and Theorem 4.5] and from [351 Theorem 6.7]. 

Proposition 2.6 (properties of u)*-integrable convex-valued multi- 
functions). In the setting of Defi,nition \2.I>\ the following hold: 

(i) Every w* -scalarly measurable multifunction with bounded values ad¬ 
mits a w*-scalarly measurable w*-almost selector. 

(ii) Every w* -scalarly integrable multifunction with w*-compact and con¬ 
vex values is w*-integrable. 

(iii) A w*-scalarly measurable multifunction T: D ^ X* with w*-com¬ 
pact and convex values is w*-integrable if and only if every w*-scalarly mea¬ 
surable, w*-almost selector ofT is w*-integrable. In this case we have that 
the w*-integral ofT is given by 

w*- j Td/i ~ |y I f ^ ‘^r I every X G S. (2.4) 
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We recall some definitions and facts from measure space theory; see, e.g., 
[T7] . Denote by AA(/i) the null ideal of S, i.e., AA(/r) := {A^ S S | fi{N) = 0}. 
A measure /r on a cr-algebra S is K-additive if for every pairwise disjoint 
family £ C T, with \£\ < k we have G S and 

where the sum is understood as supj-|-£; /r(A). The additivity 

n{fi) of la is the largest cardinal of k for which /r is K-additive, or it is oo if 
is K-additive for every k. It follows from the dehnition that K(/r) > Mi for 
every measure /r, where Mi signifies the first uncountable cardinal. A useful 
representation taken from m Proposition 521A] is 

K(/r) = min ||T| | £ C AA(/i), ^ s| . 

Denote further by dens(X) the density of the Banach space X, i.e., the 
smallest cardinal of the form \D\, where D is a dense subset of the open unit 
ball Bx of X with respect to the norm topology. 

Proposition 2.7 (equivalence of r(;*-integrable selectors). Let (D, S, yt) 

be a eomplete finite measure space, X be a Banach space, and P : D ^ X* be 
a w*-scalarly integrable multifunction with w*-compact and convex values. 
If dens{X) < k{h), then . 

Proof. By Proposition 12.61 iSp“^ is nonempty. Choose any / G . Pick 
a dense subset {a:o}a<dens(JV) from Bx and let G S with fj.{£l\Ea) = 0 be 
such that {f{uj),Xa) < s(xQ,r(a;)) for every u G Ea and a. It follows from 
K(/x)-additivity that E := nQ<dens(x) belongs to S and /x(D \ E) = 0 . 
Since P has w;*-compact values and {a:a}Q,<dens(x) is dense in Bx, the above 
inequality yields {f{co),x) < s{x,T{uj)) for every io € E and x £ X. In view 
of the rc*-compactness and the convexity of ^(^^), the separation theorem 
guarantees that /(w) G r(i^) for every cu G D. Therefore, / G 5p. □ 

If X is a separable Banach space, then dens(Al) = Mq and hence the den¬ 
sity condition in ProDosition l2. 71 obviously holds due to n{fi) > Mi. However, 
the validity of the density condition goes far beyond separability. In partic¬ 
ular, if D is a set of cardinality k while /x is a probability measure on the 
power set of D with p,{{ui}) = 0 for every cj G D, then we have that /.x is k- 
additive; see, e.g., [121 Proposition 1194]. 

It follows from Dehnition 12.II ProDosition l2.6l iil. and the above relation¬ 
ships between the sets <Sp and <Spthat in the xx;*-compact and convex¬ 
valued setting of Dehnition 12.51 we have the inclusion 

j Pd/x C w*- j Tdyi, (2.5) 

which holds as equality if dens(Al) < k(^) in view of Proposition 12.71 

Now we are ready to derive the major result of this section, which es¬ 
tablishes the xx;*-compactness and convexity of the i(;*-closure of the Gelfand 
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integral (j2.ip of multifunctions under the nonatomicity hypothesis. It im¬ 
proves the result of [241 Theorem 2] in the sense of removing the sepa¬ 
rability assumption on X and weakening the graph measurability of T in 
S 0 Borel(X*, w*) to the i(;*-measurability in the Asplund space setting. 

Theorem 2.8 (convexity of the closure of the Gelfand integral). 

Let be a complete and nonatomic finite measure space, and let X 

be an Asplund space with dens(X) < //T: ^ X* is an integrably 

bounded, w*-scalarly measurable, Gelfand integrable multifunction with w*- 

- W* 

closed values, then the set /(T) is w*-compact and convex satisfying the 
relationship I(T)^ = I{c6'^*T). 

Proof. Proceeding as in [241 Claim 3 of the Proof of Theorem 2], we conclude 

- W* 

that /(P) is w^-compact and convex. Employing now the relationship 

w*- [ co’"*rd// = /(cd’“*r) 


that is due to Proposition 12.71 and the rc*-integral representation (12.41) in 
Proposition 12.61 we get 


s{x,I{co^ r)) =s^x,r(;*- J co^ VdpLj = J s(x,co’" T)dp. 

= / s{x,T)dp,= sup / {f{uj),x)dpL 

J /g51 j 

= sup fdp,x^ = s{x,I{r)) = s ^ 


for every x ^ X, where the second equality follows from Dehnition l2.5l of the 
rc*-integrals and the forth equality uses Proposition 12.41 (where we employ 
the Asplund property). Since I{T) and I{co^*T) are w;*-compact and 

convex, we thus obtain the claimed equality /(P) = /(cd’"*r) by using 

the convex separation theorem. □ 


It has been well-recognized in the literature that the hnite-dimensional 
formulation of the Lyapunov convexity theorem fails to hold in inhnite-di- 
mensional Banach spaces without the additional closure operation; see, e.g., 
m Examples IX. 1.1 and 1.2]). Due to this, the nonatomicity of the measure 
p is insufficient to guarantee that the Gelfand integral /(P) is ic*-compact 
and convex, and thus the closure operation is inevitable in Theorem 12.81 To 
overcome this difficulty, we introduce the saturation requirement on mea¬ 
sure spaces along the lines of da [201 [231 EH], which ensures the validity of 
the Lyapunov convexity theorem with no closure operation in dual spaces 
to separable Banach spaces; see [26] )• Furthermore, the saturation property 
is also necessary for the rc*-compactness and the convexity of the Gelfand 
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integral of multifunctions as well as for the Lyapunov convexity theorem in 
separable Banach spaces; see [25] . 

Recall that a finite measure space is said to be essentially countably 
generated if its cr-algebra can be generated by a countable number of subsets 
together with measure null sets. If it is not the case, then the measure space 
under consideration is said to be essentially uncountably generated. Let 
Tie = {AnE I ^ € S} be the restriction of the cr-algebra S to a measurable 
set Li G S. Denote by L^(/i) the space of /r-integrable functions defined on 
the restricted measure space {E,T,E,ia). 

Definition 2.9 (measure saturation). A finite measure space (D,S,/r) is 
SATURATED if the space L\,{pl) is nonseparable with respect to the L^-norm 
topology for every E gT, with fi{E) > 0. 

This property, which surely implies nonatomicity, and the results related 
to it have been found as an important tool for various economic applications 
and probabilistic models; see, in particular, the papers on saturation listed 
above. Several equivalent definitions of the saturation property are known; 
see, e.g., [23]. One of the remarkable characterizations of saturation is as 
follows: a finite measure space (D, S, p) is saturated if and only if (Li, Se, p) 
is essentially uncountably generated for every E gT, with p{E) > 0. 

Proposition 2.10 (convexity of the Gelfand integral). Let (D,S,/i) 
be a complete and saturated finite measure space, and let X be a separable 
Asplund space. IfT-.Ll ^ X* is an integrably bounded and w*-scalarly 
measurable multifunction with w*-closed values, then the set I(T) is w*- 
eompact and convex satisfying the relationship I{T) = I{co'^*T). 

Proof. It follows from |19l Theorem 11] that /(T) is i(;*-compact and convex. 

Since I{co^*T) is w^-compact and convex, and /(T) C L(cd’"*r) = L(r) 
by Theorem 12.81 we obtain the equality /(T) = /(co’"*r). □ 

3 Subdifferentiation of Integral Functionals 

This section concerns integral functionals of type ()l.ip defined via the inte¬ 
gral of the integrand functions ip(x,uj) that are locally Lipschitzian in x and 
//-measurable in lo. For convenience and without loss of generality in the 
proofs below, we suppose in what follows that ip{x, •) is /i-integrable for any 
X around the reference point x. According to the discussions in Section 1, 
our goal is to derive appropriate subdifferential versions of the classical Leib¬ 
niz rule on differentiation under the integral sign, where the integration of 
set-valued subdifferential mappings is understood accordingly either in the 
Gelfand sense of Definition 12.11 or in the modified sense of the r(;*-integral 
from Definition 12.51 for convex-valued multifunctions. 
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We begin with the generalized differential constructions by Clarke [TO] for 
Lipschitz continuous functions on arbitrary Banach spaces. Given : W —>■ M 
locally Lipschitzian around x, recall first its generalized directional derivative 
at X in the direction h ^ X defined by 


4>°{x]h) 


lim sup 

X^X 

eio 


(f){x + 9h) — (j){x) 
9 


(3.1) 


A crucial property of the function h (j)°{x;h) is its automatic convexity, 
which is the source—together with the convex separation theorem—of nice 
calculus rules for it as well as for the generalized gradient (known also as 
the convexified or Clarke subdifferential) oi (p at x induced by (13.11) via the 
conventional duality scheme 

d°4>{x) := {x* G X*\ {x*,h) < (j)°{x;h) for every h G A} (3.2) 

of generating subdifferentials from directional derivatives. It is easy to ob¬ 
serve that the set d°(j){x) is nonempty, convex, and r(;*-compact in X*. Fur¬ 
thermore, the convexity of (p°{x', •) easily implies by convex separation that 
(13.ID is the support function (12.2D of the generalized gradient, i.e., we have 

4>°{x-,h) = s[h,d°4>{x)) for every/i G A. (3.3) 


Recall that (p is regular at x if the classical directional derivative 


(p'(x: h) := lim 
6i;o 


(p{x + 9h) — (p{x) 
9 


(3.4) 


exists and agrees with (13.ip . i.e., (p'{x;h) = (p°{x;h) for every h € X. The 
class of regular functions contains smooth and convex ones as well as their 
reasonable extensions and compositions; see |lf)| for the facts reviewed above. 

Having an integrand A x —>■ M in (jl.ip defined on the product 
of a Banach space A and a finite measure space (H,S,/r) and assuming 
that (/?(•, w) is locally Lipschitzian around some point x, we consider the 
generalized directional derivative (p°{x,uj; h) oi gp at x in the direction /i G A 
for each fixed w G H and then the generalized gradient d°(p{x,oj) of at x 
induced by (p°{x, oj] h) in (|3.2p . It follows from the definition of the r(;*-scalar 
measurability of the multifunction and construction (13.2p of the generalized 
gradient that the multifunction d°(p{x, •) is w;*-scalarly measurable (resp. 
r(;*-scalarly integrable) if and only if the generalized directional derivative 
function (p°{x,-]h) is measurable (resp. integrable) for each h G A. This 
surely holds when ip is regular at x, since in this case we get (p°{x,uj;h) = 
ip'{x,u};h), and thus (/?°(x,-;/i) is measurable as the pointwise limit (13.4p 
of a sequence of measurable functions. Since the multifunction d°ip{x,-) 
has convex and r(;*-compact values in A*, we proceed in what follows with 
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evaluating the generalized gradient of the integral functional (HH) at X in 
terms of the w*-integral of the generalized gradient mapping •). 

The next theorem justifies a generalized Leibniz rule in this vein for 
the case of general Banach spaces X. Its Bochner integral counterpart was 
obtained in [101 Theorem 2.7.2] for separable Banach spaces, and the sepa¬ 
rability assumption seems to be indispensable in the proof therein. 

Theorem 3.1 (generalized gradients of integral functionals in Ba¬ 
nach spaces). Let be a complete finite measure space, and let X 

be an arbitrary Banach space. Consider the integral functional I^p in dni), 
where the integrand ip: X x Ll ^ M is Lipsehitz continuous in x on some 
neighborhood U of x for every w G with an integrable Lipsehitz modulus 
being uniform on U. Assume also that (p{x,-) is measurable for every x G C/ 
and that the multifunction d°p{x,-) is w*-scalarly measurable. Then is 
locally Lipschitzian around x and we have the inclusion 

d°I^{x)Cw*- j d°(p{x,u!)dp = j d°(p{x,uj)dfj.. (3-5) 

If furthermore is regular at x for every a; G then is also regular 

at this point and (I33D holds as equality. 


Proof. Observe first that the local Lipsehitz continuity of I^p around x follows 
from the one for with the uniformly integrable Lipsehitz modulus 

on the finite measure space. Further, the w^-scalarly integrability of the 
multifunction d°p(x, ■) implies its t(;*-integrability by ProDosition l2.6l iil and 
the /i-integrability of the Lipsehitz modulus readily yields the p-integrability 
for the function lo p°{x,co;h). To verify now inclusion (13. fix x G X 
and h G X. Take any x^ —>■ x and 0 j, 0 satisfying 


i;{x-,h) 


lim ^nh') lipiXn) 

n—^oo 0^ 


By definition of the generalized directional derivative (13. ip we have 


=iim j 

^ n^oo J 0^ 

_ f pjxn + Onh, uj) - (p{Xn,Ul) 

J ^n^oo On 

< 


J (p°{x,(jj]h)dp 
= J s{h,d°ip{x,uj)^dpL 
h,w*- J d°(f{x,ui)dp^ 


where we invoke the Lebesgue dominated convergence theorem in the second 
lines and the last equality follows from Definition 12.51 of the t(;*-integral for 
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convex-valued multifunctions. Employing the separation theorem verifies 
the claimed inclusion in (13.5p . Since every selector from d°(p{x,-) is a w*- 
almost selector by (13.31) , we have the equality in (13.51) by (12.41) . The equality 
and regularity statements of the theorem can be justified similarly to the 
corresponding part in the proof of |lf)[ Theorem 2.7.2]. □ 

Let us present a direct consequence of Theorem 13.11 which gives us a 
precise extension of the classical Leibniz rule as equality for integral func¬ 
tionals on Banach spaces under an appropriate differentiability assumption 
on the integrand. Recall that a function cf): X —>■ M is strictly differentiable 
at X with its strict derivative S/(p{x) € X* if (j){x) < oo and 

fix + h) - fix) - {X(l){x),h) 

hm - — -= 0. 

h->-q \\h\\ 

X—>X 

This property implies that cj) is locally Lipschitzian around x and regular at 
this point with X(f>{x) = {V(/>(x)}; see [13 Propositions 2.2.4 and 2.3.6]. 

Corollary 3.2 (Leibniz rule for Gelfand integral functionals with 
strictly differentiable integrands). In addition to the general setting 
of Theorem 13.11 suppose that (/?(•,a;) is strictly differentiable at x for every 
a; € n with jjV(/?(x, •)]] € L^{h) t^o,t (^(x, •) is measurable for every x 
around x. Then is locally Lipschitzian around x and we have 

= J Xip{x,uj)dp., (3.6) 

where the strict derivative of tp in dMl) is taken with respect of x. 

Proof. We have from the above that in the setting of this corollary all the 
assumptions of Theorem 13.II are satisfied and the equality holds in (|3.5p with 
d°p{x,uj) = {V(/j(3:, a;)} for every a; € fl. It remains to mention that the w*- 
integral in (13.5p reduces to the Gelfand one for single-valued integrands. □ 

Next we discuss effective conditions for the validity of the w;*-scalar 
measurability assumption on the mapping d°p{x, ■) imposed in Theorem 13.11 

Proposition 3.3 (rc*-scalar measurability of the generalized gradi¬ 
ent). Assume that dens(X) < K{fi) in the setting of Theoreni A?,.l[ Then the 
generalized gradient mapping d°p{x, •) is w*-scalarly measurable. 

Proof. As discussed, it suffices to show that the measurability of the general¬ 
ized directional derivative uj p°{x,uj; h). To proceed, pick a dense subset 
{3;o}a<dens(x) of Bx and a countable dense set {9k} in M, and then choose 
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a sequence of i 0 as /c —)> oo. It follows from the density of {xQ}Q,<dens(x) 
and {0fc} and from the continuity of </?(•, w) that 


ip°{x,u}]h) = lim 

= lim 

k—^oo 


sup 

x&x+eBx, 6 s ( 0 ,£) 


sup 

«<dens (Jf) 
n&IN-.6n&(0,ek) 


if{x + 9h, cu) — ip{x, LO) 

9 

ip{{x + EkXg) + 9nh, Ul) - cp(x + EkXg, Oj) 

9n 


where the scalar function 


UJ 


sup 

a<dens(X) 

nG]N:0nG{0,ek) 


(p((x + EkXg) + 9nh, Oj) - ip{x + ekXo,,Uj) 
9n 


is measurable for each k G IN due to dens(X) x Mq < x cf. 

m Proposition 521B]. Thus it shows that the function ui (->■ ip°{x,uj;h) is 
measurable as the pointwise limit of a sequence of measurable functions. □ 


Next we proceed with establishing the extensions of the Leibniz rule 
o from 1311 Lemma 6.18] outlined in Section 1. Our standing setting in 
this part is the class of Asplund spaces X, which may be nonreflexive and 
nonseparable. Given a function (/>: X —>■ M locally Lipschitzian around x, 
recall that the limiting subdifferential (known also as the basic, general, or 
Mordukhovich one) of 0 at x is defined by 


d4>{x) := |x' 


€ X’* 


sequences Xk 


X, x. 


X* such that 


lim inf 


(j){x) - (f){Xk) - {xl,x - Xk) 


>0 


(3.7) 


||X - Xk\ 


In [30], the reader can hnd a comprehensive theory for the subdifferen¬ 
tial (13.71) in Asplund (and partly in general Banach) spaces with its various 
applications in |31] . Note that d(p{x) 0 for every locally Lipschitzian func¬ 
tion on an Asplund space, and in fact this property is a characterization of 
Asplund spaces; see m Corollary 3.25]. We have d4>{x) = {V(/>(x)} when </> 
is strictly differentiable at x, and (13.711 reduces to the subdifferential of con¬ 
vex analysis for convex functions cf). But it may be heavily nonconvex even 
for simplest nonconvex functions on M as, e.g., in the case of (p{x) = — |x| 
where d4>{0) = {—1,1}. Furthermore, it follows from |31l Theorem 3.57] 
that 


d°4>{x) = co'“*d4>{x) 


(3.8) 


whenever cj) is locally Lipschitzian around x on an Asplund space X. It is 
worth mentioning here that in spite of (in fact due to) its nonconvexity the 
limiting subdifferential (13.7p enjoys full ealeulus on Asplund spaces that is 
based on variational/extremal principles of variational analysis; see |30j . 
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Consider now the Gelfand integral m of the nonconvex subdifferential 
mapping dip{x, •): ^ X* given by 

l[d(p{x, •)) = / d(p{x, uj)dfi, 

where the notation dif{x,uj) stands for d{if{-,uj))(x). 

The following result is an extension of m Lemma 6.18] from reflexive 
and separable to general Asplund spaces X and from Bochner to Gelfand 
integration with respect to complete nonatomic measures. 


Theorem 3.4 (limiting subdifferential of integral functionals in As¬ 
plund spaces). Let (n,S,/r) be a complete and nonatomic finite mea¬ 
sure space, and let X be an Asplund space with dens(A) < K(/i). Let 
(p : X X Ll ^ W be an integrand in (HTD such that the function 
is Lipschitz continuous on a neighborhood U of x for every eo £ Ll with 
a uniformly on U integrable Lipschitz modulus, while the function (p(x, •) is 
measurable for every x £U and such that the mapping dip{x, •) is w*-scalarly 
measurable and Gelfand integrable. Then we have the inclusion 


dLr. 


x) C 


J dip{x,uj)dpL 


(3.9) 


which holds as equality provided that is regular at x for all iw £ Q. 


Proof. As shown in the proof of Theorem 13.11 the integral functional in 
(HTD is locally Lipschitzian around x under the assumptions made. More¬ 
over, the local Lipschitz continuity of with a uniformly integrable 

modulus implies the uniform boundedness of d(p{-,u}) on U and hence the 
integrable boundedness of the multifunction d(p{x, •) with ?r*-compact val¬ 
ues; see [321 Corollary 1.81]. It follows from (|3.8p that 

s[h,dip{x,uj)) = s(/i,cb“' dip{x,uj)) = s(^h,d°ip{x,io)) 


for every h £ X and uj £ Ll, which implies the w*-scalar measurability 
of dip{x, ■) and that of d°(p{x, ■) are equivalent. Further, the Gelfand in- 
tegrability of dip{x,-) yields that of d°ip{x,-), which verifies the inclusion 
(|3.5D by Theorem 13.11 On the other hand, we have by Theorem 12.81 under 

the imposed nonatomicity assumption that L{co'^*d(p{x, •)) = L{dip{x, ■)) 
Combining this with (12.4p and (13.8h leads us to the relationships 


(9/<^(x) C (9°/(^(x) Cw*- J d°ip{x,uj)dp = w*- J co'^*dip(x,uj)dp 


= Lied' 


/ LU 

dip{x,uj)dp , 


which justifies the claimed inclusion (|3.9I) . To justify the equality therein 
under the regularity of at x, we can easily derive from the definitions 
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and (j3.8jl that this regularity yields d°ip{x,Lj) = dif{x;uj) for every w € O. 
Using now the equality in (13.5p and the regularity of at x established in 
Theorem 13.II together with the equalities obtained above in the proof of this 
theorem, we arrive at the relationships 


dl^{x) = d°I, 


,{x) = w*- J d°ip{x,ui)dfj, = j d(p{x,uj)dfi 


which verify the equality in (|3.9p under the regularity of (/?(•, a;) at x. □ 


As mentioned above, the w*-scalar measurability of d(p{x, •) assumed 
in Theorem 13.41 is equivalent to the one for d°(p{x, •), and hence it holds 
under the validity of the condition dens(A) < «(//) in Proposition 13.31 in 
particular, when the space X is separable. 

The next consequence of Theorem 13.41 as well as some previous results 
presented above establishes a Leibniz-type rule for subdifferentiation of in¬ 
tegral functionals without convexification. 


Corollary 3.5 (unconvexified Leibniz rule for subdifferentiation of 
integral functionals). Let be a complete and saturated finite 

measure space, let X be a separable Asplund space, and let the integrand 
(p: X X Ll ^ M in dEU) satisfy the general assumptions of Theorem 13.41 
where the Gelfand integrability of d(p{x, ■) is automatic. Then we have 


dl^{x) C 


dip{x, u})dn 


d°ip{x, uj)dfj., 


(3.10) 


where the Gelfand integral of the subdifferential mappings is equivalent to 
the w*-integral and to the Bochner one, and where the inclusion in (13.1011 
holds as equality provided that p>{-,tv) is regular at x for every uj € Ll. 

Proof. Both conclusions in (I3.10p as well as the equality statement under 
regularity of at x follow from the combination of Proposition 12.101 

and the proof of Theorem I3.4I As mentioned above, the Gelfand integral in 
(j3.10ll agrees with the rc*-integral due to the separability of X, while both of 
these integrals of the subdifferential mappings reduce to the Bochner integral 
in separable Asplund spaces by the discussions in Remark I2.31 □ 

Remark 3.6 (separable reduction in Asplund spaces). There is the 
method of separable reduction in theory of Asplund spaces, which provides 
the possibility to elevate results involving the so-called “Prechet subdiffer¬ 
ential” (known also as the regular or viscosity subdifferential, or the presub¬ 
differential in variational analysis) of A —>■ M at x defined by 


5(()(x) := |x* € A” 


limmf ^ , 


X — X 
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from separable to general Asplund spaces. It has been used to extend a 
number of important results of generalized differentiation in the Frechet 
sense established in separable Asplund spaces to arbitrary Asplund spaces 
without any separability-like assumptions. The reader can find more infor¬ 
mation on the usage of this method, largely developed by Marian Fabian, 
in [301 Section 2.2.2] that is based on |14] : see also EH for the recent devel¬ 
opments. Remembering definition (|3.7I) of the limiting subdifferential, we 
conjecture that the results of Theorem 13.41 and Corollary 13.51 related to sep¬ 
arable Asplund spaces X can be extended to the arbitrary Asplund space 
setting by implementing the method of separable reduction. More precisely, 
we conjecture that the w^-scalar measurability assumption on d(p{x, ■) im¬ 
posed in Theorem 13.41 and satisfied in separable Asplund spaces, can be 
avoided. Proceeding in this way, we conjecture that the separability as¬ 
sumption on X can be dismissed (or seriously weakened) in Corollary 13.51 
for the Gelfand integral of the limiting subdifferential mapping. It should be 
mentioned to this end that, besides subdifferential separable reduction elab¬ 
orations, more work is needed regarding the saturation property. Indeed, 
Proposition I2.1UI used in the proof of Corollary 13.51 assumes separability. 
It has been shown quite recently [26] that the Lyapunov theorem holds in 
dual Banach spaces without any separability assumptions but with a certain 
higher-order strengthening of the saturation property. It is an interesting 
open question about the possibility to remove or relax the latter assumption 
in the nonseparable framework of Corollary 13.51 while combining with the 
method of separable reduction. 

4 Bellman Equation for Problems of Stochastic 
Dynamic Programming in Banach Spaces 

As mentioned in Section 1, our intention in this paper is to give applications 
of the obtained results on subdifferentiation of integral functionals to some 
models of stochastic dynamic programming (DP) governed by discrete-time 
systems on the inhnite horizon and mainly related to the Markov decision 
process in which current random shocks affect a probability measure over 
random shocks next period as well as a cost function and a feasible set. 
Models of this type have been investigated in finite-dimensional spaces with 
various applications to economic dynamics; see, e.g., [l2| and the references 
therein together with further references to be presented below. 

In this section we describe the basic stochastic DP model in Banach 
spaces and derive the corresponding Bellman equation for the optimal value 
function. The form of the obtained equation gives us the opportunity to em¬ 
ploy in the next section the subdifferential results for the integral functionals 
established in Section 3 in order to conduct a local sensitivity analysis and 
derive necessary optimality conditions in stochastic DP. 
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To begin with the description of the random environment of the model, 
let P be a stochastic kernel (or a transition probability measure) on a measur¬ 
able space (n, S), i.e., P: S x n —>■ [0,1] is such that P{- | w) is a probability 
measure on S for every a; G 0 and that P(A | •) is a measurable function 
on n for every A G S. Denote by w* = (wi,..., ut), t = 1,2,..a finite se¬ 
quence of random shocks in the product space D* equipped with the product 
(T-algebra S*. Given an initial shock wq € D, let P*(- | cuo) be the probabil¬ 
ity measure on S* constructed from the iterated integrals of the stochastic 
kernel P satisfying for every measurable rectangle Ai x ... x At ^ with 
its probability defined for every t = 1,2,... by 


P\Ai X ... X At I Wo) : 



wt_i)P(dwt-i 


OJt-2) ■ ■ ■ P{dLOi I Wo). 


Let X be an action space, which is assumed to be Banach in what follows. 
At each period a decision maker possesses a random cost function u : X x 
A X D —>■ R and discounts the sum of expected costs with a constant rate 
(3 G [0,1). Feasible policies are determined by a random constraint mapping 
r: A X D ^ A. Given an initial condition {x,uj) G A x D, we consider the 
stochastic DP problem on the infinite horizon described by: 


inf |u(xo,xi,wo) ld^u{xt,xt+i,uJt)P\diJ 

s.t. xt+i G r(xt,wt) for every t = 0,1,..., 
(xo,wo) = (x,w) G A X D. 



(4.1) 


Denoting by v{x,co) the inhmum value in problem (|4.ip and ranging the 
initial condition (x,w) in v(x,lo) over the whole domain A x D gives us the 
(random) optimal value function u: A x D ^ R. The assumptions made 
below ensure that the dehned value function v{x,io) is actually real-valued. 

In accordance with the choice of topologies on A, two Borel a-algebras 
arise in A: Borel (A, || • ||) is the Borel u-algebra with respect to the norm 
topology and Borel (A, w) is the Borel u-algebra with respect to the weak 
topology. As well known (see, e.g., |l5l p. 67]), in the case of separable 
Banach spaces A considered below these two notions coincide. Having this 
in mind, we say that the multifunction T : A x D ^ A is measurable if for 
every open set 1/ C A in the norm topology the inverse image (or preimage) 


T-\U) := {(x,w) G A X D I r(x,w) n C/ / 0} 


of U under T belongs to the cr-algebra Borel(A, || • ||) (g) S. 

For our subsequent study we impose the following standing assumptions 
on the initial data of the stochastic DP problem (14.ip : 

(HI) The cost function u:AxAxD—)-Ris bounded and such that 
u{-, -jw) is continuous (resp. weakly continuous) on A x A for every w G D 
while u{x,y, •) is measurable on D for every {x,y) G A x A. 
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(H2) The constraint mapping T : X x ^ X is measurable with 
compact (resp. weakly compact) values and such that r(-,a;) : X ^ X is 
continuous (resp. weakly continuous) for every a; G O while r(x, •) : X 

is measurable for every x G X. 

Denote by Cb{X x D) (resp. Cb{X^ x D)) the space of bounded functions 
(/>: X X D —>■ M endowed with the supremum norm 


sup I 

{x^lo)G.X xQ 




such that X !->■ (p{x,Lj) is continuous (resp. weakly continuous) for every 
a; G D while u i—>■ (j)(x,u}) is measurable for every x G X. Since weak 
continuity implies norm continuity, we have that Cb{Xy^ x D) C Cb{X x D) 
and that cj) G Cb{X^ x D) is a Caratheodory function with respect to the 
norm topology of X. Hence it is Borel(X, || • ||) (8) H-measurable whenever 
X is separable; see [U Lemma 4.51]. This implies that u{x,y,u}) is jointly 
measurable under the assumptions in (HI), (H2) and the separability of X. 

The next major result shows that the (optimal) value function v{x,u}) 
is a unique Caratheodory-type solution to the integral Bellman equation for 
the stochastic DP problem (14.ip involving the pointwise minimization over 
the constraint set r(x,a;). Note that the strong (resp. weak) continuity 
of the solution in the following theorem is in accordance with the strong 
(resp. weak) continuity assumptions in (HI) and (H2). Observe also that 
the form of the Bellman equation below corresponds to the discrete-time 
system in (14.11) , while its continuous-time counterpart relates to the so-called 
“viscosity solutions” of the Hamilton-Jacobi (or Hamilton-Jacobi-Bellman) 
partial differential equation; see, e.g., m with the references therein. 

Theorem 4.1 (integral Bellman equation in stochastic DP). Let X 

be a separable Banach space, and let the assumptions in (HI) and (H2) hold. 
Then there exists a unique function v in Cb{X x H) {resp. in Cb{Xw x H)) 
satisfying the following Bellman equation for every {x,uj) G X x H.- 

v{x,uj) = min < u{x,y,uj)ft / v{y,oj')P{duj' \ oj) > . (4.2) 

yerix,io) [ J J 

Proof. Define the Bellman operator T on Cb{X x H) (resp. Cb{Xw x H)) by 

{T(f>){x,u}) := min \ u{x,y,uj) + ft / 4){y,u}')P{duj' \ lo)\ . (4.3) 

yerix,io) [ J J 

To demonstrate that Tcf belongs to Cb{X x H) (resp. to Cb{Xyj x H)) for 
every (f in Cb{X x H) (resp. Cb{Xy] x H)), we apply the measurable maximum 
theorem from [H Theorem 18.19] to the objective function on the right-hand 
side in (1131), which is measurable in {x,y,uj) and continuous (resp. weakly 
continuous) in {x,y), and to the measurable multifunction T in order to 
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justify the measurability of Tcj). Picking then any w G apply the Berge 
maximum theorem from [U Theorem 17.31] to the objective function in ()4.3I) . 
which is continuous (resp. weakly continuous) in (x, y), and to the continuous 
(resp. weakly continuous) multifunction r(-,a;) with compact (resp. weakly 
compact) values. This allows us to assert that T(j)\s continuous (resp. weakly 
continuous) in x. Then the Blackwell theorem from [U Theorem 3.53] tells 
us that the Bellman operator in (|4.3I) is a contraction transformation of 
Ch{X X n) (resp. Ch{Xw x fl)) into itself. Hence for every (x,a;) G X x H this 
operator admits and a unique fixed point v = Tv solving thus the Bellman 
equation (14.2p . The verification that v is the optimal value function for the 
stochastic DP problem (|4.ip is standard; see [421 Theorem 9.2]. □ 

Define further the policy multifunction G : X x X hy 


G{x,u}) = |y G r(x,a;) v{x,uj) 


u{x,y,u;) 



v{y, oj')P{doj' 



Employing again the Berge maximum theorem ensures that G is a multi¬ 
function with compact (resp. weakly compact) values such that G{-,uj) is 
upper semicontinuous (resp. weakly upper semicontinuous) for every w G D. 
Furthermore, the measurable selection result from PQ Theorem 18.19] tells us 
that for every x £ X there is a measurable function g{x, ■) : it ^ X such that 
g{x,io) G G{x,io) whenever cj G D for the multifunction G(x, •) : it ^ X. 
Having this in mind, we say that g : X x il ^ X is a policy function for 
(j4.2p if g{x,-) is a measurable selector of G(x, •) for every x £ X. Given a 
sample path {wi}^Q in il, the policy function g generates an optimal ran¬ 
dom sequence {xt}^Q in X constructed recursively by xt+i := g{xt-,0Jt) for 
each f = 0,1,... in the original stochastic DP problem (j4.ip . 


5 Subdifferentiation of the Optimal Value Func¬ 
tion in Stochastic Dynamic Programming 

The right-hand side of the Bellman equation (|4.2p can be treated as the opti¬ 
mal value function in the deterministic problem of parametric optimization 

'd{z) ■.= m.i {^(f{z,p)\p £ M{z)'\. (5.1) 

Functions of type m, known also as marginal functions, have been investi¬ 
gated in variational analysis from the viewpoint of generalized differentiation 
with numerous applications to optimization, stability, control, and related 
topics; see, e.g., [ini EB [Ml [Ml [Ml E] and the references therein. In what 
follows we obtain new results in this direction with taking into account the 
special structure of ()4.2I) and implementing the subdifferential formulas for 
the integral functionals established in Section 3. 
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We start by establishing subdifferential results for the value function 
v{x,uj) in terms of the generalized gradient ()3.2p in Banach spaces for which 
the following local viability conditions on the policy multifunction (14.41) . 

Definition 5.1 (local viability). Let G: X x Ll ^ X be the policy multi¬ 
function with G{x,uj) 7 ^ 0 for some x G X and all uj £ Q. We say that: 

(i) G is LOCALLY LOWER VIABLE around X if for every uj £ Li there is a 
neighborhood U of x such that G{x,uj) D r(x',w) ^ 0 for every x,x' G U. 

(ii) G is LOCALLY UPPER VIABLE around X if for every ui £ LI there is 
a neighborhood U of x such that G{x,uj) C r(x',(v) for every x,x' £ U. 

Note that the local lower viability condition in Definition EHi) is a 
reasonable extension of the standard interiority condition and allows us to 
obtain the local Lipschitz continuity of the value function v{-,u:). The local 
upper viability property is used below for evaluating the generalized gradient 
of v{-,oj) and deriving necessary optimality conditions in problem (P) in its 
terms; see Theorem 15.21 As shown in Theorem 15.51 this assumption is not 
needed if we employ the limiting subdifferential in Asplund spaces. Observe 
that the local upper viability holds automatically if T is independent of x. 
It also holds under the standard interiority condition; see [U Lemma 3.1]. 

It is worth mentioning that the results below require the separability of 
the space in question, since the Bellman equation in Theorem 14. ll is derived 
under this assumption. However, the arguments employed in the proofs of 
the subsequent results do not use separability. 

To formulate the next theorem, for any given oj £Ll denote by 

gphG(-,w) := {{x,y) £ X x X\ y £ G{x,u})} 

the graph of the policy multifunction G{-,co) : X ^ X and by d°u{x,y,uj) 
the partial generalized gradient (|3.2I) of u{-,y,uj) at x G X when {y,uj) is 
fixed. The notation dyu{x,y,uj) is similar. 

Theorem 5.2 (generalized gradient of the valne function in stochas¬ 
tic DP). Let X be a separable Banach space, and let x £ X be such that 
G{x,uj) / 0 whenever u £ LI. Assume that the cost function is 

locally Lipschitzian around (x,y) for every y £ G(x,uj) with x near x and 
u £ LI and that the policy multifunction G is locally lower viable around 
X. Then the value function v{-,uj) is locally Lipschitzian around x for every 
u £LI. If furthermore G is locally upper viable around x and ifu{-,-,u}) is 
regular at {x,y) £ gphG(-,a;) for some y £ r(x,a;), then we have 

d°v{x,uj) <Z d°u{x,y,uj), uj£LI. (5-2) 

Proof. Fix {x,uj) £ X X LI and by the local lower viability of G find a 
neighborhood Ux of x such that G(x,a;) fl r(x',a;) 7 ^ 0 for every x,x' £ Ux- 


21 


Picking y € G(x,uj) D r(x',a;) for arbitrary points x,x' € Ug gives us 


v{x,uj) = u{x,y,ui) + P j v{y,u)')P{du)'\uj). 

Since v{x', uj) < u{x', y, u})+/3 J v{y, Lj')P{duj' \ u) by (|4.2I) and since u( uj) 
is locally Lipschitzian with modulus we have 

v{x',uj) — v{x,uj) < u{x',y,uj) — u{x,y,u}) < — x\\. 

Interchanging the role of x and x' above tells us that 

\v{x,uj) — v{x',uj)\ < i{ijj)'^x — x''^ whenever x,x'€Ux, 

and hence the value function v{ ■,uj) is locally Lipschitzian for every w € fl. 

Next we justify the inclusion (lOD under the additional assumptions 
made. It follows from the local upper viability of G that for every x ^ Ux 
and any given direction h £ X we have G{x,uj) C r(x + 6h,uj) when 6 > 0 
is sufficiently small. Without loss of generality choose y G G{x,uj) C r(x + 
9h,uj) for every 0 > 0 and thus get 

v{x,co) = u{x,y,ui) + P J v{y,uj')P{duj'\ uj). 

By the principle of optimality in dynamic programming we have 

v{x + Oh, ui) < u{x + 6h, y,uj) + J v{y, uj')P{duj' \ uj), 


v{x + 9h, uj) — v{x, uj) ^ u{x + 9h, y, uj) — u{x, y, uj) 


9 


9 


(5.3) 


Passing to the limit in ()5.3I1 gives us 

u{x + 9h, y, uj) — u{x, y, uj) 


lim sup 
9i0 


9 


< lim sup 

{x,y)^{x,y) 

eio 


u{x + 9h, y, uj) — u{x, y, uj) 
9 ^ 


where the notation (x, y) ^ (x, y) means that the limit {x, y) —(x, y) 

in (15.3p is taken along {x,y) G gphG(-,ti;). This shows by (13.111 that 

v°{x,uj;h) < u° [x,y,uj-, {h,0)) = u'[x,y,uj; {h,0)) 

= u'x{x,y,uj-,h) = u°x{x,y,uj-,h), 
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where u'{x,y,uj; {h,0)) (resp. u°{x,y,uj; {h,0))) denotes the (resp. general¬ 
ized) directional derivative of at {x,y) in the direction (/i, 0) € 

X X X, and where u'^{x,y,u}] h) (resp. u°{x,y,uj;h)) denotes the partial 
(resp. generalized) directional derivative of u{-,y,uj) at x in the direction 
h G X. The obtained inequality is equivalent to 

s{h,d°v{ x,uj)) < s(^h, d°u{x,y,uj)) for every hGX 

in terms of the support function (12.2p . Employing finally the separation 
theorem, we arrive at (15.21) and complete the proof. □ 

As a simple consequence of Theorem 15.21 we get the following result of 
the strict differentiability of the value function It is a signihcant 

improvement upon the known results in this direction with applications to 
optimal economic growth models (see, e.g., la El El El [271112]) , since we 
manage to remove the convexity and the submodularity assumptions and 
mitigate the interior condition in the Banach space setting. 

Corollary 5.3 (strict differentiability of the value function in stochas¬ 
tic DP). Assume in the setting of Theorem 15.21 that G is locally upper 
viable around x and that u{-,y,uj) is strictly differentiable at x G X with 
{x,y) G gphG(-,a;). Then the value function v{-,uj) is strictly differentiable 
at X and its strict derivative at x is calculated by 

Xv{x,u}) ='Vxu{x,y,uj), well. 

Proof. It immediately follows from the facts |10] that any function strictly 
differentiable at a given point is regular at this point and its generalized 
gradient reduces to the strict derivative therein. □ 

The next theorem provides necessary optimality conditions in the stochas¬ 
tic DP problem (|4.ip formulated in the form of the stochastic Euler inclu¬ 
sion involving the r(;*-integral from Definition 12.51 and the construction of 
the generalized normal cone [TO] to a subset C of a Banach space defined by 

-- w* 

N°{z]C) := a5°dist(^;G) at z G C (5.4) 

Q :>0 

via the generalized gradient of the Lipschitz continuous distance function to 
a set given by dist(z; C) := inf{||c — z|| c G C}. The set C is regular at z G C 
if the associated distance function is regular at this point. 

Note also that the theorem below imposes the integrability of the Lips- 
chitzian modulus of u(-, -, 0 ;) from Theorem 15.21 with respect to the measure 
P{- I cu), which precisely means that f £(uj')P(duj' | w) < 00 for such a 
modulus £(uj) whenever to G Cl, y G G{x,u}), and x near x. 
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Theorem 5.4 (stochastic Euler inclusion in Banach spaces). Let X 

be a separable Banaeh space. In addition to the assumptions of Theorem \b.2{ 
suppose that the Lipsehitz modulus £(uj) ofu{-, -, 0 ;) is integrable with respect 
to a complete probability measure P{- \ oj), that u{-, -, 10 ) is regular at {x,y), 
and that G is locally upper viable around x. Then for every policy function 
g : X X Ll ^ X we have the following stoehastic Euler inclusion: 

0 e d°u{x,y,uj) +I3w*- J d°u{y,g{y,uj'),uj')P{du;'\uj) 

+N° {y;T{x, w)), w € O. 

Proof. Since y is a local optimal solution to the constrained minimization 
problem in (14.21) . we have from |10l Corollary to Proposition 2.4.3] that 

Oedy + P j v{-,uj')P{du' I {x,y) + N° {y-,T{x,u:)), 

which implies by the calculus rules from cni Proposition 2.3.1 and Propo¬ 
sition 2.3.3] the validity of the inclusion 

dy + P j v{-,u:')P{duj' \uj)^ {x,y) 

C d°u{x, y,uj) + /3d°(^J v{-,u}')P{duj' \ cv)^ (y). 

On the other hand, the generalized Leibniz rule from Theorem 13.11 yields 

v{-,u}')P{du}' \ C w*- J d°v{y,u}')P{du}' \ lo). 

Taking now any policy function g and using Theorem 15.21 tell us that 

d°v{y,uj') C d°u[y,g{y,uj'),uj') for every lv'gLI. (5.6) 

Integrating this inclusion with respect to P{- \ oo) and combining it with the 
inclusions above, we arrive at the stochastic Euler inclusion (|5.5p . □ 

We now proceed with deriving necessary optimality conditions for the 
stochastic DP problem (14. ip by using the limiting subdifferential construc¬ 
tions in Asplund spaces. In this way we obtain a more refined inclusion for 
optimal solutions in comparison with the stochastic Euler one ()5.5p under 
somewhat different assumptions. We need the following additional defini¬ 
tions while referring the reader to |30] for more details and discussions. 

Given a; G D, we say that r(-,a;) : X ^ X is Lipschitz-like around 
(x, y) & X X X (or has Aubin’s pseudo-Lipschitz property) if there are 
neighborhoods U of x and E of y as well as modulus £{u}) > 0 such that 

T{x.,uj) CiV (ZT{x\oj)-\-l{u})\\x — x'\\Bx for every x,x'^U. 
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A policy multifunction G{-,uj) : X =1 X is inner semicontinuous at (x,y) € 
gphG(',u;) if for every sequence —>■ x there is a sequence of G G{xk,uj) 
that contains a subsequence converging to y, where the convergence is un¬ 
derstood in the norm topology of X. 

Similarly to but a bit differently from ()5.4h . define the basic/limiting 
normal cone [30] to a subset C of a Banach space Z by 

N{z]G) := a5dist(z;C') at z € C (5-7) 

o>0 

via the limiting subdifferential (I2Z3) of the distance function. We have 
N°{z‘,G) = co^ N(z;G) provided that the space Z is Asplund and that 
G is locally closed around z; see |3Ul Theorem 3.57]. 

Our first result in this direction provides a Gelfand integral condition 
for the limiting subdifferential of the random value function on the 

Asplund action space X that generally holds without imposing either the 
regularity assumption on the cost function u(-, ■,uj) or the local upper via¬ 
bility assumption on the policy multifunction G. 

Theorem 5.5 (Gelfand integral relations for the limiting subdif¬ 
ferential of the random value function in stochastic DP). Let X be 

a separable Asplund space, and let be locally Lipschitzian around 

(x,y) G gphG(-,u;) for every uj G Q so that its Lipschitz modulus i{uj) 
is integrable with respect to a complete and saturated probability measure 
P{- I uj). Assume also that the constraint mapping r(-,a;) is of closed graph 
and Lipschitz-like around (x,y) and that the policy multifunction G{-,uj) is 
locally lower viable around x and inner semicontinuous at (x,y). Then there 
exists X* G X* such that we have the inclusion 

{0} ^ ^ J dv{y,uj')P{duj' \ uj) 

-FX((x,y);gphr(-,cj)), w G 0. 

If furthermore the function is regular at {x,y) and the graph of the 

mapping r(j(x) := r(x,a;), x G X, is also regular at this point for every 
UJ G then (15.81) is equivalent to the inclusions 

0 G dyu{x,y,uj) +/3 j dv{y,uj')P{duj'\uj) + N{y;T{x,uj)), 
x* G d^u{x,y,uj) + N{x]T~^{y)), u G Ll. 

Proof. It is shown in Theorem 15.21 that the value function v{-,uj) and the 
objective function u{-,-,uj) + P J v{-,uj')P{duj' \ tv) in problem (|4.ip are lo¬ 
cally Lipschitzian around x under the general assumptions of the theorem. 
Thus all the assumptions of [341 Corollary 3] obtained for the parametric 


(x*,0) G du{x,y,uj) -|- 
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optimization framework m are satisfied in our setting (|4.2H . which allows 
us to find a dual element x* € X* such that 


(x*,0) G (9 +/3 j v{-,u}')P{doj' \ oj)^ (x, y)+fV((x, y); gphr(-, w)). 

It follows from the subdifferential sum rule of m Theorem 2.33] that 
d •, w) + /3 j v{-,J)P{dJ I (x, y) 

C du{x,y,oj) + {0} X I3d(^j v{-,uj')P{duj' \ uj^{y) ■ 

Employing further the Leibniz rule obtained in Corollary 13.51 yields 

d(^ J v{-,uj')P{du}' I ui)^{y) C J dv{y,uj')P{du}' \ uj) 

and thus yerifies (15.81) under the general assumptions made. 

Inyoking now the additional regularity assumptions giyes us 


du{x, y, Lo) = dxu{x, y, lo) x dyu{x, y, lo) 

by |3r)l Corollary 3.44] and its corresponding counterpart for the limiting 
normal cone (|5.7p that follows from the proof of |30l Corollary 3.17]. Hence 
we arriye at ()5.9I) and thus complete the proof of the theorem. □ 


Now we can deriye from Theorem 15.51 and Theorem 15.21 the following 
necessary optimality conditions for problem (SH) entirely yia its initial data, 
where the refined stochastic Euler inclusion significantly improyes the one 
from Theorem 15.41 under somewhat different assumptions. 


Corollary 5.6 (enhanced stochastic Euler inclusion in Asplund spaces). 

In addition to the general assumptions of Theorem 15.51 we suppose that the 
cost function «(•,•, w) is regular at (x,y) G gphG(-,a;) and that the policy 
multifunction G is locally upper viable around x. Then there exists x* G X* 
such that for every policy function g : X x It ^ X the following enhanced 
stoehastic Euler inclusion is satisfied: 


0,fi J d^u{y,g{y,uj'),u}')P{duj' \ui) 
+A((x,y);gphr(-,a;)), w G H. 

If furthermore the set gphT^j is regular at (x, y) for every lo G id, then (|5.1UI) 
is equivalent to the two inclusions: 

G dyu{x,y,uo) + fi J dxu{y,g{y,uj'),uj^)P{du}' \ u) + N{y;T{x,uj)), 

X* G da;u{x,y,to) + N{x]r-^{y)), lo G il. 


{x*,0) G du{x,y,Lo) + 
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Proof. Integrating inclusion ()5.6p with respect to the saturated probability 
measure P{- \ iv) and then employing Corollary 13.51 yield the relationships 

J dv{y,uj')P{dij' I w) = J d°v{y,uj')P{duj' \ u) 

C j d°u{y,g{y,uj'),u}')P{du}' \u}) = j dxu{y,g{y,Lo'),uj')P{duj' \ uj). 

Substituting the obtained inclusion into (15.81) giyes us (j5.10l) . The equiya- 
lence of the latter to the last relationships in the corollary under the addi¬ 
tional regularity assumption follows from the proof of Theorem 15.51 □ 


Similarly to Remark 13.61 we conjecture that deyeloping the method of 
separable reduction would allow us to overcome the separability assumption 
on the action space X in Theorem 15.51 and its consequences inyolying the 
limiting subdifferential constructions in Asplund spaces. 

The results obtained aboye in this section address the case of general 
random constraint mappings T : X x II ^ X. In the yast majority of 
applications the mapping T is not arbitrary while is giyen in some structural 
form. Thus in structural cases it had become crucial to eyaluate the normal 
cone in question yia the initial data of structural constraint mappings, which 
largely relates to the existence of strong/full calculus for generalized normals 
and subgradients. Such calculus is ayailable for the limiting constructions in 
Asplund spaces; see [30] and the references therein. In particular, this full 
calculus makes it possible to eyaluate the limiting normal to the graph of T as 
in (|5.8I) and (|5.inp (which relates to the coderivative [30] of T) and represent 
it in terms of the problem data in structural problems. We refer the reader 
to |3ll Section 4.3] for calculations of the coderiyatiye for yarious classes of 
set-yalued mappings T and to [331 El] for their particular applications to 
KKT systems in nonlinear programming. 

Finally in this paper, we eyaluate the limiting subdifferential of the ran¬ 
dom optimal yalue function v{-,io) for the stochastic DP problem (|4.ip en¬ 
tirely yia the problem data for the case of the random constraint mapping 
r(a:,a;) defined in the nonlinear programming (NLP) form by 


r(x,u;) := \ y ^ X 


ipi{x,y,uj) <0, i = 1,... ,m, 

(/9j(x, y, a;) = 0, i = m -|- 1,..., m -|- r 


(5.11) 


where the functions y?, : X x X x D —>■ R, i = 1,..., m -|- r, are such 
that are strictly differentiable at the reference point {x,y) with 

their strict deriyatiyes Vipi{x, y,Lv) while ipi{x, y, ■) are measurable in accor¬ 
dance with assumption (H2) on P. Let us first recall the classical constraint 
qualification in nonlinear programming. Giyen {x,y) € X x X, denote the 
collection of active constraint indices by 


I{x,y,uj) := {i £ {l,...,m + r}\ ipi{x,y,u}) = O}. 
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It is said that the parametric Mangasarian-Fromovitz constraint qualification 
(MFCQ) holds at {x,y) if for every a; € we have the conditions: 

(i) The equality constraint gradients Vipm+i{x, y, co), ..., Vifm+rix, y, w) 
are linearly independent in X x X. 

(ii) There exists ^ £ X x X such that {Xifi{x,y,u}),^) = 0 for i = 

m + 1,..., m + r and IfiJipfix, y, a;), < 0 for i G {1,..., m} fl I(x, y, to). 

Denote the objective function ^Min (14.1|) by 

<f{x,y,oj) := u{x,y,oj) + fi j v{y,J)P{dJ\u:). (5.12) 

Taking A := (Ai,..., \m+r) £ and assuming that u{-, -^oj) is strictly 

differentiable at (x, y) as u £ D, consider the set of Lagrange multipliers 


A{x,y,ui) := < 

X £ 

m+r 

Alyipix, y, w) + ^ XiXyLpiix, y, w) = 0, | 


< 

Ai > 0, Xupfix, y, w) = 0, i = 1,..., m J 


where we get by (j5.12D and the classical Leibniz rule that 

Xyg){x,y,La) = Xyu{x,y,u) + fi j Vv{y,uj^)P{duj^ \ u). 

Observe that the set A{x, y, co) describes the Lagrangian stationary condition 
XyL{x,y, \,co) = 0 via the Lagrange function 

m-\-r 

L{x,y,X,uj) := ip{x,y,uj) + ^ Xtipfix, y, u). 

i=l 

The following property is needed for the formulation of the next theorem. 
We say that a function f : D C X ^ X is locally upper Lipschitzian ai x £ D 
if there exist real numbers y > 0 and > 0 such that 

||/(x) — /(x)|| < l\\x — x|| for every x £ By{x) PI D. 

Such a function is said to be a local upper Lipschitzian selector of a given 
multifunction F : D ^ X around (x, y) £ gph F if /(x) = y and /(x) G F{x) 
for every x in some neighborhood of x. 

Now we are ready to present the last result of the paper. Observe that 
its hnite-dimensional version in the inclusion form can be found in [39] for 
problems of type (|4.ip with only the inequality constraints in (15.lip given 
by smooth functions under the standard interiority condition and the linear 
independence of the gradients Vyipi{x,y,uj), i £ I{x,y,u}), which is surely 
stronger than the parametric MSCQ imposed below. 
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Theorem 5.7 (limiting subgradients of the value function in stochas¬ 
tic DP with NLP constraint mapping). Let X be a separable Asplund 
space, and let the constraint multifunction T : XxLl ^ X of (BU) is given in 
the NLP form (|5.11l) . Assume that u{-, ■,u}) and -jw), i = 1,... ,m + r, 
are strictly differentiable at {x,y) € gphG(-,a;), that the policy multifunc¬ 
tion G is inner semicontinuous at {x,y), and that the parametric MFCQ is 
satisfied at this point. Then we have the inclusion 

{ m+r ^ 

Xxu{x, y, w) + ^ XiXxTi{x, y,uj)>, to 

i=l ) 

Furthermore, the equality holds therein provided that G{-,uj) admits a local 
upper Lipschitzian selector around the reference point {x,y). 

Proof. The Bellman equation (|4.2H ensures the representation of v{x,uj) as 
a marginal function m under the separability assumption. It remains 
to use the subdifferential results from [341 Corollary 4], which work in the 
general framework of (I5.ip in Asplund spaces and ensure the validity of both 
statements of the theorem under the imposed assumptions. □ 

Acknowledgements. The authors are gratefully indebted to Marian 
Fabian, Ali Khan, Lionel Thibault, and anonymous referee for their valuable 
remarks that allowed us to improve the original presentation. 


References 

[1] C. D. Aliprantis and K. C. Border, Infinite Dimensional Analysis: A Hitch¬ 
hiker’s Guide, 3rd edn.. Springer, Berlin, 2006. 

[2] C. D. Aliprantis, G. Camera and F. Ruscitti, Monetary equilibrium and the 
differentiability of the value function, J. Econom. Dynam. Control 33 (2009), 
454-462. 

[3] R. Amir, Sensitivity analysis of multisector optimal economic dynamics, J. 
Math. Econom. 25 (1996), 123-141. 

[4] K. Askri and C. Le Van, Differentiability of the value function of nonclassical 
optimal growth models, J. Optim. Theory Appl. 97 (1998), 591-604. 

[5] L. M. Benveniste and J. A. Scheinkman, On the differentiability of the value 
function in dynamic models of economics, Econometrica 47 (1979), 727-732. 

[6] J. M. Bonnisseau and C. Le Van, On the subdifferential of the value function 
in economic optimization problems, J. Math. Econom. 25 (1996), 55-73. 

[7] B. Cascales, V. Kadets and J. Rodriguez, Measurability and selections of multi¬ 
functions in Banach spaces, J. Convex Anal. 17 (2010), 229-240. 

[8] B. Cascales, V. Kadets and J. Rodriguez, The Gelfand integral for multi-valued 
functions, J. Convex Anal. 18 (2011), 873-895. 


29 




[9] N. H. Chieu, The Frechet and limiting subdifferentials of integral functionals 
on the spaces Li{n,E), J. Math. Anal. Appl. 360 (2009), 704-710. 

[10] F. H. Clarke, Optimization and Nonsmooth Analysis, John Wiley & Sons, New 
York, 1983. 

[11] M. Ciith and M. Fabian, Asplund spaces characterized by rich families and 
separable reduction of Frechet subdifferentiability, J. Funct. Anal. 270 (2016), 
1361-1378. 

[12] A. Dasgupta, Set Theory: With an Introduction to Real Point Sets, Birkhauser, 
New York, NY, 2014. 

[13] J. Diestel and J. J. Uhl, Jr., Vector Measures, Amer. Math. Soc., Providence, 
Rl, 1977. 

[14] M. Fabian and B. S. Mordukhovich, Separable reduction and extremal princi¬ 
ples in variational analysis. Nonlinear Anal. 49 (2002), 265-292. 

[15] S. Fajardo and H. J. Keisler, Model Theory of Stochastic Processes, A. K. 
Peters, Ltd., Natick, MA, 2002. 

[16] W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity 
Solutions, Springer, New York, 1993. 

[17] D. Fremlin, Measure Theory, Vol.5: Set-Theoretic Measure Theory, Part I, 
Torres Fremlin, Cochester, 2008. 

[18] E. Giner, Subdifferential regularity and characterizations of the Clarke sub¬ 
gradients of integral functionals, J. Nonlinear Convex Anal. 9 (2008), 25-36. 

[19] M. Greinecker and K. Podczeck, An exact Fatou’s lemma for Gelfand integrals 
by means of Young measure theory, to appear in J. Convex Anal. 

[20] D. Hoover and H. J. Keisler, Adapted probability distributions, Trans. Amer. 
Math. Soc. 286 (1984), 159-201. 

[21] A. D. Ioffe and V. L. Levin, Subdifferentials of convex functions, Trans. 
Moscow Math. Soc. 26 (1972), 1-72. 

[22] A. Jourani and L. Thibault, Noncoincidence of approximate and limiting sub¬ 
differentials of integral functionals, SIAM J. Control Optim. 49 (2011), 1435- 
1453. 

[23] H. J. Keisler and Y. N. Sun, Why saturated probability spaces are necessary, 
Adv. Math. 221 (2009), 1584-1607. 

[24] M. A. Khan, On the integration of set-valued mappings in a non-reflexive 
Banach space, II, Simon Stevin 59 (1985), 257-267. 

[25] M. A. Khan and N. Sagara, Maharam-types and Lyapunov’s theorem for vector 
measures on Banach spaces, Illinois J. Math. 57 (2013), 145-169. 

[26] M. A. Khan and N. Sagara, Maharam-types and Lyapunov’s theorem for vector 
measures on locally convex spaces with control measures, J. Convex Anal. 22 
(2015), 647-672. 

[27] T. Kim, Differentiability of the value function: A new characterization, Seoul 
J. Econom. 6 (1995), 257-265. 


30 


[28] D. Maharam, On homogeneous measure algebras, Proc. Natl. Acad. Sci. USA 
28 (1942), 108-111. 

[29] A. Maugeri, Equilibrium problems and variational inequalities, in Equilibrium 
Problems: Nonsmooth Optimization and Variational Inequality Methods (F. 
Giannessi et al., eds.), Nonconvex Optim. Appl. 58, pp. 187-205, Kluwer, 
Dordrecht, 2001. 

[30] B. S. Mordukhovich, Variational Analysis and Generalized Differentiation, I: 
Basic Theory, Springer, Berlin, 2006. 

[31] B. S. Mordukhovich, Variational Analysis and Generalized Differentiation, II: 
Applieations, Springer, Berlin, 2006. 

[32] B. S. Mordukhovich and N. M. Nam, Variational stability of marginal functions 
via generalized differentiation, Math. Oper. Res. 30 (2005), 800-816. 

[33] B. S. Mordukhovich, N. M. Nam and H. M. Phan, Variational analysis of 
marginal functions with applications to bilevel programming, J. Optim. Theory 
Appl. 152 (2012), 557-586. 

[34] B. S. Mordukhovich, N. M. Nam and N. D. Yen, Subgradients of marginal func¬ 
tions in parametric mathematical programming. Math. Program. 116 (2009), 
369-396. 

[35] K. Musial, Approximation of Pettis integrable multifunctions with values in 
arbitrary Banach spaces, J. Gonvex Anal. 20 (2013), 833-870. 

[36] J. P. Penot, Image space approach and subdifferentials of integral functionals, 
Optimization 60 (2011), 69-87. 

[37] R. R. Phelps, Gonvex Functions, Monotone Operators and Differentiability, 
2nd edn., Springer, Berlin, 1993. 

[38] K. Podczeck, On the convexity and compactness of the integral of a Banach 
space valued correspondence, J. Math. Econom. 44 (2008), 836-852. 

[39] J. P. Rincon-Zapatero and M. S. Santos, Differentiability of the value function 
without interiority assumptions, J. Eeonom. Theory 144 (2009), 1948-1964. 

[40] R. T. Rockafellar, Convex integral functionals and duality, in Gontributions 
to Nonlinear Functional Analysis (E. H. Zarantonello, ed.), pp. 215-236, Aca¬ 
demic Press, New York, 1971. 

[41] L. Schwartz, Radon Measures on Arbitrary Topological Spaces and Gylindrical 
Measures, Oxford University Press, London, 1973. 

[42] N. L. Stokey, R. E. Lucas, Jr. and E. C. Prescott, Recursive Methods in Eco¬ 
nomic Dynamics, Cambridge, Massachusetts, Harvard University Press, 1989. 

[43] Y. N. Sun and N. C. Yannelis, Saturation and the integration of Banach valued 
correspondences, J. Math. Econom. 44 (2008), 861-865. 

[44] L. Thibault, On subdifferentials of optimal value functions, SIAM J. Gontrol 
Optim. 29 (1991), 1019-1036. 

[45] G. E. F. Thomas, Integration of functions with values in locally convex Suslin 
spaces, Trans. Amer. Math. Soc. 212 (1975), 61-81. 

[46] M. Valadier, Convex integrands on Souslin locally convex spaces, Pacif. J. 
Math. 59 (1975), 267-276. 


31 


